Comprehensive Training and Evaluation on Deep Reinforcement Learning for Automated Driving in Various Simulated Driving Maneuvers

Developing and testing automated driving models in the real world might be challenging and even dangerous, while simulation can help with this, especially for challenging maneuvers. Deep reinforcement learning (DRL) has the potential to tackle complex decision-making and controlling tasks through learning and interacting with the environment, thus it is suitable for developing automated driving while not being explored in detail yet. This study carried out a comprehensive study by implementing, evaluating, and comparing the two DRL algorithms, Deep Q-networks (DQN) and Trust Region Policy Optimization (TRPO), for training automated driving on the highway-env simulation platform. Effective and customized reward functions were developed and the implemented algorithms were evaluated in terms of onlane accuracy (how well the car drives on the road within the lane), efficiency (how fast the car drives), safety (how likely the car is to crash into obstacles), and comfort (how much the car makes jerks, e.g., suddenly accelerates or brakes). Results show that the TRPO-based models with modified reward functions delivered the best performance in most cases. Furthermore, to train a uniform driving model that can tackle various driving maneuvers besides the specific ones, this study expanded the highway-env and developed an extra customized training environment, namely, ComplexRoads, integrating various driving maneuvers and multiple road scenarios together. Models trained on the designed ComplexRoads environment can adapt well to other driving maneuvers with promising overall performance. Lastly, several functionalities were added to the highway-env to implement this work. The codes are open on GitHub at https://github.com/alaineman/drlcarsim-paper.Comment: 6 pages, 3 figures, accepted by the 26th IEEE International Conference on Intelligent Transportation Systems (ITSC 2023

Martini 3 : a general purpose force field for coarse-grained molecular dynamics

The coarse-grained Martini force field is widely used in biomolecular simulations. Here we present the refined model, Martini 3 (http://cgmartini.nl), with an improved interaction balance, new bead types and expanded ability to include specific interactions representing, for example, hydrogen bonding and electronic polarizability. The updated model allows more accurate predictions of molecular packing and interactions in general, which is exemplified with a vast and diverse set of applications, ranging from oil/water partitioning and miscibility data to complex molecular systems, involving protein-protein and protein-lipid interactions and material science applications as ionic liquids and aedamers.Peer reviewe

K-core in random graphs

A graph G=(V,E) is a mathematical model for a network with vertex set V and edge set E. A Random Graph model is a probabilistic graph. A Random Geometric Graph is a Random Graph were each vertex has a location in a space χ. We compare the Erdos-Rényi random graph, G(n,p), to the Random Geometric Graph model, RGG(n,r) where, in general we use r=c / (n^(-1/d)), with dimension d. It is known that for p = λ*/n the k-core has a first-order phase transition in G(n,p) where λ* is the critical value for the k-core. The k-core is a global property of a graph. The k-core is the largest induced subgraph where each vertex has at least degree k. We suggest by simulations and a supportive proof that for the RGG-model a first-order phase transition not plausible. A inhomogeneous extension of the RGG-model with a vertex weight distribution T is a Geometric Inhomogeneous Random Graph model (GIRG). We also prove why some heavy-tailed (i.e. power-law) distributions almost surely have a k-core, when the amount of vertices v, which have weights greater than the square root of n, is greater than k. Furthermore, we rephrase from known literature how using a fixed equation for a branching process is a useful tool for analysing the existence of a k-core. In particular, the critical value for the 3-core is recovered using the probability of a binary tree embedding in branching processes, with the root having at least 3 children.Applied Mathematic

Monitoring evapotranspiration over the Alpilles test site by introducing remote sensing data at various spatial resolutions into a dynamic SVAT model

International audienc

Comparing patient reported abdominal pain between patients treated with oxaliplatin-based pressurized intraperitoneal aerosol chemotherapy (PIPAC-OX) and primary colorectal cancer surgery

Oxaliplatin-based pressurized intraperitoneal aerosol chemotherapy (PIPAC-OX) is an emerging palliative treatment for patients with unresectable colorectal peritoneal metastases. Previously, our study group reported that patients experienced abdominal pain for several weeks after PIPAC-OX. However, it is unknown how this compares to abdominal pain after regular colorectal cancer surgery. To provide some perspective, this study compared the presence of abdominal pain after PIPAC-OX to the presence of abdominal pain after primary tumor surgery. Patient reported abdominal pain scores (EORTC QLQ-CR-29), from two prospective, Dutch cohorts were used in this study. Scores ranged from 0 to 100, a higher score represents more abdominal pain. Abdominal pain at baseline and at four weeks after treatment were compared between the two groups. Twenty patients who underwent PIPAC-OX and 322 patients who underwent primary tumor surgery were included in the analysis. At baseline, there were no differences in abdominal pain between both groups (mean 20 vs. 18, respectively; p?=?0.688). Four weeks after treatment, abdominal pain was significantly worse in the PIPAC group (39 vs 15, respectively; p?<?0.001; Cohen's d?=?0.99). The differential effect over time for abdominal pain differed significantly between both groups (mean difference: 19 vs -?3, respectively; p?=?0.004; Cohen's d?=?0.88). PIPAC-OX resulted in significantly worse postoperative abdominal pain than primary tumor surgery. These results can be used for patient counseling and stress the need for adequate analgesia during and after PIPAC-OX. Further research is required to prevent or reduce abdominal pain after PIPAC-OX.Trial registration CRC-PIPAC: Clinicaltrails.gov NCT03246321 (01-10-2017)